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ABSTRACT 

The one observation per cell two-way items by 
examinees random effects analysis of variance (ANOVA) with all error 
components zero is considered. The estimated variance components are 
expressed as functions of the inter-item covariance matrix and the 
inter-examinee covariance matrix. These expressions show that under 
the random effects model if the inter-item and inter-examinee 
covariance matrices are unconstrained then both the number of items 
and the number of examinees must approach infinity for the estimates 
of any of the variance components to be consistent. However, if these 
two covariance matrices are constrained so that each has homogeneous 
variances and convar iances , then consistent estimates of the variance 
components can be obtained without both the number of items and the 
number of examinees simultaneously approaching infinity. (Contains 
seven references.) (Author) 
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A Note on a Relationship Between Covariance Matrices 
and Consistently Estimated Variance Components 



David J. Woodruff 



Abstract 



The one observation per cell two-way items by examinees random 
effects ANOVA with all error components zero is considered. The 
estimated variance components are expressed as functions of the inter- 
item covariance matrix and the inter-examinee covariance matrix. These 
expressions show that under the random effects model if the inter-item 
and inter-examinee covariance matrices are unconstrained then both the 
number of items and the number of examinees must approach infinity 
for the estimates of any of the variance components to be consistent. 
However, if these two covariance matrices are constrained so that each 
has homogeneous variances and covariances, then consistent estimates 
of the variance components can be obtained without both the number of 
items and the number of examinees simultaneously approaching infinity. 
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A Note on a Relationship Between Covariance Matrices 
and Consistently Estimated Variance Components 

In applications of the random effects ANOVA model to measure- 
ment data, usually little if any consideration is given to the covariance 
structure of the data — the focus is on the variance components. The 
purpose of this note is to explore the relationship between the covariance 
structure of the data and the variance components and to demonstrate 
that when the covariance structure of the data has Scheffes (1959, p. 
264) compound symmetric form, then consistent estimates of the 
variance components can be obtained more simply than when the 
covariance structure of the data is unconstrained. This simplification 
under compound symmetry has an important advantage for the practical 
application of the model. Hocking, Green, and Bremer (1989) developed 
the random effects ANOVA model with the assumption of compound 
symmetry for the covariance structure, and they presented diagnostics 
for deciding on the appropriateness of the model under this covariance 
structure, but they did not motivate the usefulness of the compound 

symmetry assumption as this note does. 

Consider the I by J data matrix Y = ji/yj that results from 

administering / items to J examinees. Two covariance matrices can be 

computed from Y: the J by J inter-examinee covariance matrix 
S e = { s iU’fj}Jjj, =l (computed using I - 1 as the divisor) and the I by I 

inter-item covariance matrix = |s J -(i,i')J / / (computed using J-l as 

the divisor). The i or j subscript associated with the elements of a sample 
covariance matrix denote the index over which the elements are 
computed, for example, 
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sjd.n = z (y y - y t .)(y i3 - ye)/ ( J - i)- 

The data matrix Y will be modeled using a one observation per cell 
two-way random effects ANOVA model. A complete and detailed 
development of this model may be found in Scheffe (1959, Chap. 7) 
where the model equation is given, the variance component parameters 
are defined, and method of moments variance component estimators are 
developed as functions of observed mean-squares. See also Searle (1971, 
Chap. 9) or Hocking (1985, Chap. 10). Note, however, that for simplicity 
and without loss of generality all error component values are assumed to 
be zero so that the error variance equals zero. Furthermore, no 
distributional assumptions are made about the observations or the 
model components. 

Using results given by Hocking et al. (1989), the method of 
moments variance component estimators given by Scheffe (1959 Chap. 7) 
also can be expressed as functions of the two sample covariance 
matrices: 
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( 3 ) 




i= i t=i v>i 





I 1 ( 1 - 1)/2 



(4) 



Note that the interaction variance component estimator can be expressed 
as a function of either or S 

A parameterization of the random effects ANOVA model in terms of 
covariance matrices gives an interesting perspective on this model. Two 

underlying countably infinite dimensional parametric covariance 
matrices, X 0 = {<T 0 (j,/)}~ and ^ = K(i,i')}“, =1 . can be postulated. 

Conditional on any random sample of I items, let X^ denote an I by I 

inter-item parametric covariance matrix whose elements are a subset of 
the elements of X^. Conditional on any random sample of J examinees, 

let X^ denote a J by J inter- examinee parametric covariance matrix 
whose elements are a subset of the elements of X 0 . Assume that 
X^ and X 0 satisfy the compound symmetry structure of Scheffe (1959, p. 
264) so that 
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where I is the identity matrix and J is a matrix of all ones and the 
dimensions of I and J are countably infinite. It then follows that X^ is 
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completely determined by the two parameters and y n which are the 
same for any selection of I items; hence, Z^ is invariant under any 
selection of I items. Similarly, Z^ is completely determined by the two 
parameters Oq and y e which are the same for any selection of J 
examinees; hence Z^ is invariant under any selection of J examinees. 

This invariance under compound symmetry has an important effect on 
the consistency of the variance component estimators. 

The situation is similar but not identical to joint estimation of IRT 
models where distinct parameters are associated with each item and 
each examinee, and both the number of items and the number of 
examinees must simultaneously approach infinity to obtain consistent 
estimates of the model parameters (Haberman, 1977). In the 
unconstrained random effects ANOVA model, a set of parameters (a 
variance and covariances) is associated with each item and each 
examinee, and both the number of items and the number of examinees 
must simultaneously approach infinity to obtain consistent estimates of 
the variance component parameters which are functions of the item and 
examinee variances and covariances. However, if the inter-item and 
inter-examinee covariance matrices are constrained to have the 
compound symmetric form, then the number of variances and 
covariances is fixed at four, and consistent estimates of the variance 
component parameters can be obtained without both I and J 
simultaneously approaching infinity. This will now be demonstrated. 

As already noted, Scheffe (1959, Chap. 7) defines the variance 
component parameters and develops method of moments estimators for 
these parameters. The variance component estimators given in 
equations (1) through (4) are algebraically equivalent to Scheffe's method 
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of moments estimators and so are consistent estimators because they are 
method of moment estimators (Serfling, 1980). Because and are of 

countably infinite dimension, when no constraints are placed on the 
structures of JL n and there are a countably infinite number of 

moments (variances and covariances) to be estimated, and so both I and 
J must go to infinity to obtain consistency. More specifically, 
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The purpose of this note is to show that with the compound 
symmetry structure imposed on Z^ and Z 0 the consistency of the 

variance component estimators does not depend on both I and J 
simultaneously approaching infinity. Because of the invariance of Z^ 

for any selection of J examinees under compound symmetry for Z 0 , it 
follows that for any random sample of J examinees: 
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and so only I need increase without bound to obtain consistent estimates 
of these two variance components. Because of the invariance of Z^ for 

any selection I items under compound symmetry for Z^, it follows that 
for any random sample of / items: 
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and so only J need increase without bound to obtain consistent 
estimates of these two variance components. Note that the interaction 
variance component can be consistently estimated as either I or J 
approach infinity under compound symmetry. 
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In the practical application of the two-way random effects ANOVA 
model (including the items by examinees situation), it is sometimes the 
case that it is expensive or impractical to increase the number of levels 
measured on the first factor but relatively inexpensive to increase the 
number of levels measured on the second factor, and the purpose of the 
study is to get an accurate estimate of the variance due to the second 
factor or to the interaction. If the compound symmetry assumption is 
not satisfied, then increasing the number of levels measured on the 
second factor is insufficient to ensure a more accurate estimate of the 
second factor variance component or the interaction variance component. 
The number of levels of the first factor would also have to be increased, 
at greater expense, to ensure more accurate estimates of the second 
factor variance component or the interaction variance component. 

Hocking (1985, Chap. 10) can be consulted for the compound 
symmetric covariance structure formulation of the two-way random 
effects ANOVA model when the error variance is non-zero and there is 
more than one observation per cell. Woodruff (1993) discusses reliability 
estimation and other psychometric issues under this model. 
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